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Abstract. Consider a system in which players at nodes of an underlying 
graph G repeatedly play Prisoner's Dilemma against their neighbors. 
The players adapt their strategies based on the past behavior of their 
opponents by applying the so-called win-stay lose-shift strategy. This 
dynamics has been studied in [Kit94,DGG+02,MR06]. 
With random scheduling, starting from any initial configuration with 
high probability the system reaches the unique fixed point in which all 
players cooperate. This paper investigates the validity of this result under 
various classes of adversarial schedulers. Our results can be summarized 
as follows: 

— An adversarial scheduler that can select both participants to the 
game can preclude the system from reaching the unique fixed point 
on most graph topologies. 

— A nonadaptive scheduler that is only allowed to choose one of the 
participants is no more powerful than a random scheduler. With 
this restriction even an adaptive scheduler is not significantly more 
powerful than the random scheduler, provided it is "reasonably fair" . 

The results exemplify the adversarial scheduling approach we propose as 
a foundational basis for the generative approach to social science [Eps07] . 

Keywords: evolutionary games, self stabilization, discrete dynamical sys- 
tems, adversarial analysis 

1 Introduction 

Evolutionary game theory [Wci95] and agent-based simulation in the social sci- 
ences [BW00,TJ06,GT05] share the object of study and a significant set of con- 
cerns, and are largely distinguished only by the method of choice (mathematical 
reasoning versus computational experiments). In particular, the two areas deal 
with fairly similar models (see, e.g. [Wil06,BEM05]). A particular class of such 
models assumes a large population of agents located at the vertices of a graph. 

* corresponding author. Email: gabrielistrate@acm.org 



Agents interact by playing a fixed game, and update their behavior based on the 
outcome of this interaction (according to a pre-specificd rule) . 

How docs one interpret properties of such systems, be they obtained through 
mathematical analysis or computational simulations ? A possible answer is that 
results characterizing dynamical properties of such models provide insights (and 
possible explanations) for features observed in "real-world" social dynamics. For 
instance, the primary intuition behind the concept of stochastically stable strate- 
gies in evolutionary game theory is that a small amount of "noise" (or, equiva- 
lently, small deviations from rationality) in game dynamics can solve the equi- 
librium selection problem, by focusing the system on one particular equilibrium. 
Similarly, in agent-based social theory, Epstein [Eps07,Eps99] (see also [AE96]) 
has advocated a generative approach to social science. The goal is to explain a 
given social phenomenon by generating it using multiagent simulations'*. Related 
concerns have been recently voiced throughout analytical social science, with a 
particular emphasis on mechanism-based explanations [Hed05,HS06]. 

Given that game theory and agent-based simulation are emerging as tools 
for guiding political decision-making (see e.g. [BEM05,ECC+04,EGK+04,TRA]), 
it is important to make sure that the conclusions that we derive from these 
techniques are robust to small variations in model specification^. 

In this paper we consider the effect of one particular factor that can affect the 
robustness of results of agent-based simulations and evolutionary game theory: 
agent scheduling, i.e. the order in which agents get to update their strategies. 
Many models in the literature assume one of two popular alternatives: 

— synchronous update: every player can update at every moment (in either 
discrete or continuous time); this is the model implicitly used by "large 
population" models in evolutionary game theory. 

— uniform matching: agents are vertices of a (hyper)graph. At each step we 
choose a (hyper)edge uniformly at random and all players corresponding to 
this hyperedge are updated using the local update function. 

histead of postulating one of these two update mechanisms, we advocate 
the study of social dynamics under an approach we call adversarial scheduling. 
We exemplify adversarial scheduling by studying, in such a setting, the Iterated 
Prisoners' Dilemma game with win-stay losc-shift strategy. This dynamics (orig- 
inally motivated by the colearning model in [ST94]) have received substantial 
attention in the game-theoretic literature[Kit94,DGG+02,MR06]. 

We are, of course, far from being the first researchers to recognize the cru- 
cial role of scheduling/activation order on the properties of social dynamics (to 

* cf. [Eps07], Chapter 1: "if you didn't grow it [the social phenomenon, n.n.], you 
didn't explain its emergence". 

^ The importance of stability has been previously recognized in the literature, for in- 
stance, in [AE96] (pp. 35), where emergent phenomena are defined as stable macro- 
scopic patterns arising from local of interaction of agents" . 



only give two examples, see [HG93,Axt00]). However, what we advocate is a 
(somewhat) more systematic approach, based on the following principles: 

(i) Start with a "base case " result P , stated under a particular scheduling model. 

(ii) Identify several structural properties of the scheduling model that impact the 
validity of P. Ideally, these properties should be selected by careful exami- 
nation of the proof of P, which should reveal their importance. 

(iii) Identify those properties (or combinations of properties) that are neces- 
sary/sufRcient for the validity of P. Correspondingly, identify properties 
that that are inconsequential to the validity of P. 

(iv) The process outlined so far can continue by recursively applying steps (i)- 
(iii). In the process we may need to reformulate the original statement in 
a way that makes it hold under larger classes of schedulers, thus making it 
more robust. The precise reformulation(s) normally arise from inspecting the 
cases when the proof of P fails in an adversarial setting. 

The intended benefits of the adversarial scheduling approach are multiple 
(see [Ist06] for further discussions). The aim of this paper is to show that these 
benefits do not come at the expense of mathematical tractability: at least for 
one nontrivial dynamics, adversarial scheduling (as outlined in the five point 
approach above) is feasible and can lead to interesting results. 

2 Preliminaries 

First we define two classes of graphs that we will frequently consider in this 
paper. The line graph in, n > 1 consists of n vertices vi,...Vn and edges 
{vi,Vi+i), 1 < i < n — 1. We will use Line to denote the set of all line graphs. 
A star graph Starn, n > 1, is the complete bipartite graph Ki n- We will use 
Star to denote the set of all star graphs. 

2.1 Basic Model: Prisoner's Dilemma Vifith Pavlov dynamics 

Next we describe the basic mathematical model for Prisoner's Dilemma with 
Pavlov dynamics; see [Kit94,MR06,DGG"''02] for additional discussion. We are 
given an undirected graph G{V,E), \V\ ~ n and \E\ = m. Each vertex v V 
represents an agent. Each agent has a label from the set {0, 1}. These labels 
denote the strategies that the players follow: can also be equivalcntly viewed 
as cooperation and 1 can be viewed as defection. Without loss of generality, we 
assume that G is connected, otherwise the dynamics will reduce to independent 
dynamics on the connected components of G. Wc will also assume that the graph 
contains at least two edges. 

Time will be discrete. At time < = all nodes are assigned a label from {0, 1}. 
At each subsequent step, certain nodes/agents change their label (strategy) ac- 
cording to the rules given below. We will use xt{v) to denote the label of node 



V. At each step t + 1 an edge e = {u, v) is selected according to some rule and 
the states of u and v are updated as follows: 

X(^t+i){u) ^ (xtiu) +xt{v)) (mod 2) 

X(t+i)(w) ^ {xt{u) + xt{v)) (mod 2) 

We will use Xt to denote the vector {xt{vi), . . . Xt{vn)) representing the states of 
the nodes vi, . . .Vn - This will be sometimes referred to as the global configuration. 
Sometime we will omit the subscript t for case of exposition and its value will 
be clear from context. With this terminology, each step of the dynamics can be 
viewed as a global update function F. It takes as input an element e = (vi, Vj) G 
X = (x{vi), . . . x{vn)) and returns the next global Y = . . . y(w„)) 

given as follows: Vwfe, s.t. k ^ i and k ^ j, y(vk) = x{vk)', and y{vi) = y{vj) = 
{x(vi)+x{vj)) (mod 2). In this case Y is said to be reachable from X in one step. 
A global configuration X is said to be a fixed point if Ve G E, F(X, e) = X. 
It is easy to see the dynamical system studied here has a unique fixed point 
= (0, ...,0). Following dynamical systems literature, a configuration X is 
called a Garden of Eden configuration if the configuration is not reachable from 
any other configuration. For the rest of this paper, we will use X, Y, . . . to 
denote global configurations. An instance of the Prisoner's Dilemma with Pavlov 
dynamics (PDPD) can thus be represented as a (G, /), where G is the underlying 
interaction graph and / is the local function associated with each node. In the 
remainder of this paper, since / is always fixed, an instance will be specified 
simply by G. 

2.2 The base-case result 

The following property of the PDPD is easily seen to hold under random match- 
ing: for all interaction graphs G with no isolated vertices the system converges 
with probability 1 — o(l) to the "all zeros" configuration (henceforth denoted 0) 
With a slight abuse of convention, we will refer to this event as self-stabilization. 
This will the statement we will aim to study in an adversarial setting. 

We will also be interested in the convergence time of the dynamics. Under 
random scheduling Dyer et al. [DGG+02] prove that the number of steps needed 
to self-stabilize is O(rilogn) on C„ (the simple cycle on n nodes) and exponential 
in n on Kn (the complete graph on n nodes) . The convergence time was further 
investigated by Mossel and Roch [MR06]. 

2.3 Types of scheduler 

A schedule S is specified as an infinite string over E, i.e. S G E* . Given a schedule 
S = (ei, 62, . . . , et, . . .), the graph G and an initial configuration I, the dynamics 
of the system evolve as follows. At time t = the system is in state I. At time 
t, we pick the edge from S. Call this edge e*. If the configuration at the 



beginning of time i is X then the configuration Y at the beginning of time t + 1 
is given by Y i^(X, et). The iterated global transition function F* is defined 
as 

(ei, 62, . . . , e^, . . .)) = F*iFiI, ei), (e2, . . . , e^, . . .)) 

We say that a system G self stabilizes for a given initial configuration I 
and a schedule S = (ei, . . . , et, . . .)^ if 3t > 1 such that the system start- 
ing in I reaches the (unique fixed point) configuration after t time steps, i.e. 
<— (ei, . . . , et)). G is said to self stabilize for a schedule 5' if G even- 
tually reaches a fixed point when started at any initial configuration /, i.e. 
V/, F{I, S) — > 0. Conversely, a schedule S can preclude self stabihzation of 
G if 3/ such that F{I, S) does not ever reach 0. Given a set of schedules, a 
scheduler is simply an algorithm (possibly randomized) that chooses a schedule. 
The schedulers considered here are all polynomial time algorithms and the set 
of feasible schedules are described below. 

Schedulers can be adaptive or non-adaptive. An adaptive scheduler decides 
on the next edge or node based on the current global configuration. A non- 
adaptive scheduler decides on a schedule in advance by looking at the graph (and 
possibly the initial configuration). This schedule is then fixed for remainder of 
the dynamic process. One particularly restricted class of non-adaptive schedules 
is a fixed permutation of nodes/edges repeated periodically and independent of 
the initial state of the system. For node updates, this is the model employed in 
sequential dynamical systems [BHM+03,BEM05]. 

There are several distinctions that can be made, concerning the power of a 
scheduler. The first one concerns the number of players that the scheduler is able 
to choose. There are two possibilities. 

— An edge-daemon (or edge-scheduler) is able to choose both players of the 
interacting pair. In other words, an edge daemon constructs S by selecting 
edges ei e i? in some order. 

— A node-daemon (or node-scheduler) can choose only one of the players. We 
can let this player choose its partner. A natural model is to consider the 
case where partner is chosen uniformly at random among the neighbors of 
the first player. In such a case, we say that the node-scheduler model has 
random choice, or random-choice node-scheduler. 

2.4 Summary of results 

Our results can be summarized as follows: 

— Not surprisingly, some amount of fairness is a necessary to extend self- 
stabilization in an adversarial setting. 

— The power given to the scheduler makes a big difference in whether or not 
the system self stabilizes: 

® Note that d and ej in the sequence need not be distinct 



(i) if the scheduler can exogenously choose both participants in the game 
then (Theorems 2,3) it can preclude convergence on most graphs, even 
when bounded by fairness constraints. 

(ii) On the other hand schedulers that allow a limited amount of endogeneity 
in agent interactions^ , by only choosing one of the participants are no 
more powerful than the random scheduler (Theorem 4) when nonadap- 
tive, and are not significantly more powerful (Theorem 5) when adaptive 
but "reasonably fair" . 

We also investigate experimentally (in Section 7) the convergence time of 
the colearning dynamics for a few nonadaptive schedulers, in a case when the 
convergence time for the random scheduler is known rigorously. 

2.5 Related work 

Our approach is naturally related to the theory of self-stabilization of distributed 
systems [DolOO]. Multi-agent systems, like the ones considered in the evolutionary 
game dynamics, have many of the characteristics of a distributed system: a num- 
ber of entities (the agents) capable of performing certain computations (changing 
their strategies) based on local information. Randomized models of this type (in- 
cluding the model we study in this paper) have been in fact recently considered 
in the context of self-stabilization [FMP05]. There are, however, a number of 
differences. First, in self-stabilization the computational entities (processors) are 
capable of executing a wide-range of activities (subject to certain constraints, 
for example the requirement that all processors run the same program, in the 
context of so-called uniform self- stabilizing systems). The goal of such systems 
is to achieve a certain goal (legal state) in spite of transient errors and malicious 
scheduling. In contrast, in our setup, there is no "goal", the computations are 
fixed, and restricted to steps of the evolutionary dynamics. The only source of 
uncertainty arises from the scheduling model. A second difference is in the na- 
ture of the update rule. Usually, in self-stabilizing systems there is a difference 
between enabled processors, that intend to take a step, and those that indeed 
take it. Such a notion is not so natural in the context of game-theoretic models. 

As mentioned earlier, the dynamics, can be easily recast in the context of Pris- 
oner's Dilemma: let 1 encode "defection" and encode "cooperation" . Then the 
update rule corresponds to the so-called win-stay, lose-shift [Pos97], or Pavlov 
strategy. This rule specifies that agents defect on the next move precisely when 
the strategy they used in the last interaction was different from the strategy 
used by the other player. It was the object of much attention in the context 
of Iterated Prisoner's Dilemma [Axc84], [NS93],[Axe97]. Related versions of the 
dynamics have an even longer history in the Psychology literature, where they 

The importance of this property has been recognized [Vri06] in the agent-based 
simulation literature 



were proposed to model emergence of cooperation in situations where players do 
not know precisely the payoffs of the game in which they arc participating, and 
might even be unaware they are playing a game: Sidowski [Sid57] has proposed 
the "minimal social situation" (MSS) , a two-person experiment representing an 
extremely simple form of interaction between two agents. MSS was first viewed 
as a game by Thibaut and Kelley [TK59] who called it "Mutual Fate Control" . 
An explanation for the empirical observations in [TK59] was proposed by Kelley, 
Thibaut, Radloff and Mundy [HKM62], that raised the possibility that players 
were acting according to the Pavlov dynamics. MSS was generalized to mul- 
tiplayer games by Colman et al. [ACT91,Col05], who obtained mathematical 
characterization for the emergence of cooperation. 

3 Fairness in scheduling 

A necessary restriction on schedulers we will be concerned with is fairness. In 
self-stabilization this is usually taken to mean that each node is updated infinitely 
often in an infinite schedule. We will also consider notions of bounded fairness. 
A natural definition is the following: 

Definition 1. Let b > 1. A scheduler that can choose one item among a set of 
m elements is (worst-case) 6- fair if for every agent x, no other agent is scheduled 
more than b times between two consecutive times that x is scheduled. 

It is easy to see that a 1-fair edge scheduler chooses a fixed permutation 
of edges uses this as a periodic schedule. A 1-fair node daemon selects a fixed 
permutation of nodes and for each node selects a random neighbor and repeats 
the same permutation (with possibly different partners) periodically. 

The fact that we want to investigate properties of the random scheduler 
suggest investigating fairness of probabilistic schedulers. For such schedulers, 
the worst-case fairness in Definition 1 is far too restrictive. 

Definition 2. A probabilistic scheduler is weakly fair if for any node x and any 
initial schedule y, the probability that x will eventually be scheduled, given that 
the scheduler selected nodes according to y is positive. 

The random scheduler is weakly fair. Not every scheduler is weakly fair, and 
a scheduler need not be weakly fair to make the system self-stabilize. On the 
other hand, the base-case result does not extend to the adversarial setting when 
weak fairness is not required: 

Theorem 1 The following are true: 

(i) There exists an edge scheduler that is not weakly fair and that makes the 
system self- stabilize no matter what its starting configuration is. 



(ii) For any graph G there exists an edge/node scheduler that is not weakly fair 
and that prevents the system from self-stabilizing on some initial configura- 
tion. 

Another restatement of Theorem 1 is that weak fairness is necessary to pre- 
clude some "degenerate" schedulers like the ones we construct for the proof of 
point (ii). 

Proof, (i) Consider an edge scheduler that works as follows: 

— Choose an edge e that has not yet self-stabilized, i.e. at least one of its 
endpoints is 1. 

— Turn nodes of e to by playing e twice. 

— The scheduler never schedules e subsequently. 

(ii) Consider a node (edge) scheduler that repeatedly schedules the same node 
(edge). It is easy to see that the system does not self-stabilize unless the 
graph consists of a single edge (a star in the case of a node scheduler when 
the center node is the scheduled one). 

Definition 3. A probabilistic scheduler is 0(/(n))-node fair w.h.p. if the fol- 
lowing condition is satisfied while the system has not reached the fixed point: For 
any schedule W (call its last scheduled node x), every node y and every e > 
there exists > such that, with probability at least 1 — e node y will be 
scheduled at most ■ f{n) times before x is scheduled again.. 

A scheduler is boundcdly node fair w.h.p. if it is 0{f{n))-node fair w.h.p. 
for some function f{n). 

We emphasize the fact that Definition 2 applies to edge schedulers as well, 
when a node x is considered to occur at stage t if some edge containing x is 
scheduled at that step. 

With these definitions we have: 

Theorem 2 Let S be a (node or edge) weakly fair probabilistic scheduler such 
that the following result holds: for any initial configuration, the probability that 
the system self- stabilizes tends to one. Then the scheduler is boundedly node fair. 

Proof. We will consider both node and edge schedulers at the same time. Let e > 
and let T = T(e, n) be an integer such that, no matter in which configuration 
S we start the system, the probability that the system does not self-stabilize 
(taken over all the coin tosses of the scheduler) is at most e. 

Consider any state S of the system after a node x has be scheduled, and 
assume that S is not the absorbing state 0. Run the system for T steps. The 
probability that the system does not self-stabilize is at most e. On the other 
hand, if some node y is not played at all during the T steps then the system has 
no chance to self-stabilize. It follows that the maximum number of times a given 
node can be scheduled before x is scheduled again is at most T — I. 



4 The Power of Edge-Schedulers 



From now on we will restrict ourselves to boundedly fair schedulers. This section 
aims to show that edge-schedulers are too powerful. Indeed, it is easy to show 
that there exist graphs on which even 1-fair edge-schedulers can prevent self- 
stabilization. The following two results provide a modest improvement, showing 
that even 2-fair edge-daemons on any graph are too strong : 

Theorem 3 Let G be an instance of PDPD. Then there exists an initial con- 
figuration I and a 2-fair edge- scheduler S that precludes self- stabilization on G 
starting in configuration I. 

Proof. Consider a sequence of edges eo,...ek (with repetitions allowed) such 
that 

~ every edge of G appears in the list. 

— for every i = 0, . . . , /c, e,; and e^+i have exactly one vertex in common (where 
Cfc+i = eo). 

— Every edge appears in the sequence at most twice. 

We will show that an enumeration F{G) with these properties can be found 
for any connected graph with more than one edge. Such a sequence specifies in a 
natural (via its periodic extension) a 2-fair edge daemon. It is easy to see that the 
only states that can lead to the fixed point are the fixed point and states leading 
to it in one step: such a state has exactly two (adjacent) ones. But such a state 
cannot be reached from any other state according to the previously described 
scheduler, since the edge that would have been "touched" immediately before 
has unequal labels on its extremities, which cannot be the case after updating 
it. 

We now have to show how to construct the enumeration F(G). First we give 
the enumeration in the case graph G is a tree. In this case we perform a walk 
on G, hsting the edges as follows: suppose the root r is connected, via vertices 
vi, . . . ,Vk to subtrees Ti, . . . , T^. Then define recursively 

F{G) = (f, ti)F{Ti){ti,v){v, t2)F{t2){t2,v) . . . {tk-i,v)iv, tk)F{Tk){tk,v), (1) 

where, if the list of edges thus constructed contains two consecutive occur- 
rences of the same edge we eliminate the second occurrence. 

Consider now the general case of a connected graph G, and let S{G) be a 
spanning tree in G. Edges of G belong to two categories: 

(i) Edges of the spanning tree S{G). 

(ii) Edges in E{G) \ E{S{G)). 

Define F{G) as the list of edges obtained from F{S{G)) in the following 
way: whenever F{S{G)) first touches a new vertex w in G insert the edges in 



E{G) \ E{S{G)) adjacent to w (in some arbitrary order); continue then with 
F{S{G)). It is easy to see that each edge e is Usted at most twice in F{G). To 
prove this consider the two cases, e G E{S{G)) and e £ E{G) \ E{S{G)). The 
statement follows in the first case by the recursive definition 1. In the second 
case it follows by construction, since a nontree edge is visited only when one of 
its endpoints is first touched in E{S{G)). 

Even the most restricted edge-schedulers, 1-fair edge-schedulers, are able to 
preclude self stabilization on a large class of graphs. To see that define 

(i) Qi to be the class of graphs G that contain a cycle of length at least four. 

(ii) Q2 to be the class of graphs G that contain no cycles of length at least four 
and m, the number of edges of G is even. 

(iii) to be the class of trees with n ^ Ak vertices. 

Theorem 4 Let G be a connected graph in QiU Q2 ^ Ga- Then there exists an 
initial configuration on G and a 1-fair edge-schedule S that is able to forever 
preclude self- stabilization on G. 

In other words, connected graphs for which the system self-stabilizes for all 
1-fair schedulers have an odd number of edges and all their cycles (if any) have 
length 3. 

Proof. Define Ai_j ~ {^kf) a. n x n matrix over Z2 by 

[ , otherwise. ^ ' 

Suppose we represent configurations of the system as vectors in . 

Taking one step of the dynamics on an arbitrary configuration X with the 
scheduled edge being {i,j) leads to configuration X = Aij ■ X, where matrix 
Aij is given by 

A, J = /„ + A J + Z\j- (3) 

Indeed, the only nondiagonal elements of matrix Aij that are nonzero are in 
positions {i,j) and {j,i). This means that all elements of a configuration X in 
positions other than i, j are preserved under multiplication with Aij. It is easy 
to sec that labels in positions i, j change according to the specified dynamics. 

Consider a graph G with m edges, E{G) — {{ii, ji), . . . , {im,jm)}- The action 
of a 1-fair edge schedule S (specified by permutation tt of {1,...,to}) on a 
configuration X corresponds to multiplication of X by 

Tt{S) ~ ■ ^i,r[2] ,i,r[2] ' • ■ • " ^i,r [m] j'lr [m] ' W 

An edge-schedule cannot prevent self-stabilization implies that starting at 
any initial configuration, the system reaches a fixed point. In other words, VI G 



Z2 3A; e N s.t.[7r(S)]'^ -1 = 0. Since the number of vectors I is finite, this is 
equivalent to saying that 3ki e N, VI, [7r(S)]'^i - X = 0. Equivalently this means 
that [7r(S)]'^ = 0, i.e. matrix 7r[S] is nilpotent. Thus, what we want to show is 

Lemma 4. For any graph G there exists a schedule S such that the correspond- 
ing matrix 7r[S] is not nilpotent. 

Consider now an arbitrary ordering of vertices in G and let tt be the permu- 
tation corresponding to the induced lexicographic ordering of edges of G (where 
an edge is seen as an ordered pair, with the vertex of lower index appearing 
first). 

It is easy to see that 

■ ^i^^i = [0 , otherwise. 

Indeed, if Aij = (amf2)m,n>i, then the only wat for some element Cm,n of the 
product Ai j ■ Aj^^i to be nonzero is that there exists at least one term dm'^p ■ d'^^n^ 
that is nonzero. But this is only possible for = (rn,p) and {p,n) = {k,l), 
in other words for m — i, n — I and p = j = k, which immediately yields 
equation (5). 

Let us now consider the integer matrix n[S] obtained by interpreting equa- 
tions (3) and (4) as equations over integers. Because integer addition and multi- 
plication commute with taking the modulo 2 value, matrix 7r[S'] can be obtained 
by applying reduction modulo 2 to every clement of n[S]. 

Let Pi_j = Aij + Aj^i. From the definition of matrix n(S) in equation (4) 
and the definition of matrices Ai^ in equation ( 3) we see that 7r[S] is a sum of 
products, each term in a product corresponding to either a Pij or to the identity 
matrix. Thus 

n[s] = i+ (n^M.i.^.w)- (6) 

07^SC{l,...,m} fees 

Consider the directed graph G obtained from G by duplicating every edge 
{i,j} of G into two directed edges {i, j), (j, i) in G. Label every edge e £ E{G) by 
the (unique) integer k such that e = {iir[fc]j J7r[fc]}j £^nd apply the same labelling to 
the two oriented versions of edge e in G. Then equation (5) shows that nonzero 
products of matrices A arc in bijective correspondence to directed paths of length 
two with increasing labels, when read from the starting to the end node of 
the path. Inductively generalizing these observations to all sets S we see that 
products in (6) are nonzero exactly when they specify a directed path in G (i.e. 
a path in G) from a vertex /c to a vertex I with increasing labels when read from 
k to I, in which case they are equal to Ak,i. 



Therefore n[S] ~ I + C, where C = (cij) is given by 




# of paths from i to j with increasing labels, if such paths exist 
0, otherwise. 



(7) 



The matrix 7r[S] is, of course, obtained by reducing modulo 2 the elements 
of II[S]. A well known result in linear algebra^ is that the the characteristic 
polynomial of a nonzero nilpotcnt matrix A is x". Thus, one strategy to show that 
a given matrix 7r[S] is not nilpotcnt is to make sure that for some p, < p < n, 
the sum Sp of its principal minors of order p is non-zero. This is equivalent to 
making sure that the sum of the corresponding minors of the associated integer 
matrix is odd. The proof consists of three cases: 

Case (a) G € Qii We will prove the following 

Lemma 5. There exist two permutations a\ and a2 with corresponding matrices 
over integers Ai = n[<7i] and A2 = n[ij2] such that 



Given Lemma 4, the proof of Lemma 3 for Case (a) follows since matrices 
7r[(7i] and 7r[(72] cannot both be nilpotcnt. This is true since the trace{Ai) and 
trace{A2) have different parities. We will prove Lemma 4 using a multistcp 
argument, combining the conclusions of Lemmas 3-5 below. Consider first the 
following "basic" graphs: K^, A'3 A A'3 (the graph obtained by merging two 
triangles on a common edge), and C„, n > 4. 

Lemma 6. The conclusion of Lemma 4 is valid for the "basic" graphs. 

Proof. By the previous result on the value of coefficients Cij the value of the 
trace of a matrix A can be easily computed from the number of cycles with 
increasing labels. Also, note the following: 

— Any cycle C contributes a one to at most one Ci^i, for some vertex i appearing 
in C. This is because of the restriction on the increasing labels, who might 
be verified for at most one node of the cycle. 

— Moreover, any triangle contributes a 1 to exactly one ctj. Therefore, in con- 
sidering the trace of matrix A triangles add the same quantity irrespective 
of permutation, and can thus be ignored. 

This observation leads to a simple solution when the underlying graph is a 
simple cycle C„, n > 4, or the graph Kr^ A K^. Note that these graphs contain 
a unique cycle C of length atleast 4. Consider an ordering of the edges of this 
cycle, corresponding to moving around the cycle. We will create two labellings 

* justified eis follows: a classical result states that the characteristic and the minimal 
polynomial of a matrix have the same roots (with different multiplicities). But it is 
easy to see that the minimal polynomial of a nilpotent matrix is x'' for some k < n. 



trace{A2) = {trace{Ai) + I) (mod 2). 



corresponding to this ordering. The first one assigns labels 1 to |C| in this order. 
The other labelling assigns labels 1, 2, . . . , |C| — 2, \C\, \C\ — 1 in this order. It 
is easy to see that the first ordering contributes a 1 to exactly one diagnonal 
element, while the second one does not contribute a 1 to any element. Hence the 
traces of the corresponding matrices differ by exactly 1. 

For graph K4 we first label the diagonal edges by 5 and 6. There are three 
cycles of length 4 in the graph K4 ~ one that uses no diagonal edges, the other 
two using them both. For the outer cycle consisting of no diagonal edges, we 
consider the two orderings described in the previous case on the outer cycle C4. 
This as before shows that the traces of the corresponding matrices differc exactly 
by 1. Next, note that irrespective of the labelling of the non-diagonal edges, the 
two cycles containing the diagonal edges cannot be traversed in increasing label 
order, so they do not contribute to the trace of the associated matrix. Therefore, 
the result follows for graph K4 as well. This completes the proof of Lemma 3. 

Lemma 7. Let G be a graph and let G2 he a subgraph of G induced by a subset 
of the vertices in G. If the conclusion of Lemma 3 holds for G2, then it holds for 
G. 

Proof. Extend a permutation of the edges in G2 to a permutation of the edges 
in G via a fixed labelling of the edges in E{G) \ E{G2) such that 

(i) The index of any edge with both ends in G2 is strictly smaller than the index 
of all edges not in this class. 

(ii) The index of any edge with exactly one end in G2 is strictly larger than the 
index of any edge not in this class. 

The trace of the resulting matrix is determined by the cycles with strictly 
increasing labels. There arc several types of such cycles: 

(i) Cycles in G \ G2 . Whether such a cycle can be traversed in increasing label 
order does not depend on the precise labelling on edges of G2 as long as the 
conditions of the extension are those described before. 

(ii) Cycles containing some edges in G2, as well as additional edges from G \ G2. 
Because of the restriction we placed on the labelings, the only such cycles 
that can have increasing labels are the triangles with two vertices in G2 and 
one vertex in G \ G2. Since there exist an unique way to "read" a triangle in 
the increasing order of the edge labels, their contribution to the total trace 
is equal to the number of such triangles, and does not depend on the precise 
labeling of edges in G2, as long as the restriction of the labeling is met. 

(iii) Cycles entirely contained in G2. 

Let now cri , CT2 be labelings on G2 verifying the conclusion of Lemma 2 and 
oT, CT2 extensions to G verifying the stated restriction. The conclusion of the pre- 
vious analysis analysis is that a difference in the parity of the traces of matrices 
corresponding to the labelings ai on G2 directly translates into a difference in 
the parity of the traces of matrices corresponding to labelings al on G. 



Finally, we reduce the case of a general graph to that of a base case graph 
via the following result. 

Lemma 8. Let G be a graph that contains a cycle of length > 4 and is minimal 
(any induced subgraph H does not contain a cycle of length > 4 any more). Then 
G is one of the "basic " graphs from Lemma 3. 

Proof. Let G be minimal with the property that it contains a cycle of length 
> 4, let n be the number of nodes in G and let C be a cycle of length > 4 in G. 
Because of minimality, C contains all the vertices of G (otherwise G would not 
be minimal, since one could eliminate nodes outside C). Thus C is a Hamiltonian 
cycle. If no other edge is present then we get the cycle C„. Moreover, no other 
edge can be present unless n = 4 (otherwise G would contain a smaller cycle of 
length > 4 and thus would not be minimal). In this case the two possibilities are 
Ki and K3 A K3. 

Case (b) G £ Q2- Consider the ordering <sum on the edges of G so that 
{hi] <sum {k,l} when either 



In this case Ck^k = for every k, except when k is the middle-index vertex of 
a triangle (i.e. a triangle with vertex labels i,j, k such that i < k < j). 

We infer that si ~ trace{A) is congruent (mod 2) to n plus the number of 
triangles in G, that is to m + 1 (mod 2) (where m is the number of edges of G). 

Case (c) G G Gs'- Consider the sum S2 of principal minors of size 2 of A. In 
a tree there can be at most one path between two nodes. Since we are counting 
paths with increasing labels, the only way for Oi.j = aj^i = 1 to hold is that 
vertices i and j be adjacent. But in this case the corresponding minor is zero. It 
follows that S2 is the number of sets of different nonadjacent vertices in G, that 
is 



One might suspect that Theorem 2 (ii) extends to all graphs, thus strenght- 
ening the statement of Theorem 1 to 1-fair daemons. This is not the case: Let 
the line graph Lg be an instance of PDPD. Then for all 1-fair edge- schedulers 
S and for all initial configurations I the system self-stabilizes starting at I . We 
verified this statement via computer simulation, by running PDPD for all 6! 
1-fair daemons. A result that rendered this experiment computationally feasible 
is the state-reduction technique highlighted in the proof of (ii): to prove self- 
stabilization we only needed to consider those initial configurations with exactly 



i-\-j<kJrl 



or 



i + j = k + I and min{i, 7} < minj/c, I}. 




if n = 4fc. 



one 1. Thus we had to run 6x6! simulations. It is an open problem to find all 
graphs for which this happens. However, Theorem 2 (ii) shows that the class of 
such graphs is really limited. 

5 Nonadaptive node-schedulers 

As we saw, even 1-fair edge schedulers are able to prevent self-stabilization. What 
if we only allow the scheduler to choose one of the nodes ? In this section we 
study the Prisoners dilemma with Pavlov dynamics when adversaries are 1-fair 
node-schedulers. Because only one of the nodes of the scheduled edge is chosen 
by the adversary and the other one is chosen randomly, the self-stabilization of 
the system is a stochastic event. 

Theorem 5 Let Starn be an instance of PDPD. Then Sn self- stabilizes with 
probability 1 against any 1-fair scheduler. 

Proof. One can assume, without loss of generality, that the first node to be 
scheduled is the center (labeled 0) and the rest of the nodes are scheduled in the 
order 1,2,..., n. Indeed, if the center was scheduled later in the permutation 
of nodes, it is enough to prove self-stabilization from the configuration that 
corresponds to first running the system up to just before the center is scheduled, 
and then viewing the run as initialized at the new configuration, and with a new 
periodic schedule (that now starts with node 0). As for the order in which the 
other nodes get scheduled, by relabelling the nodes we may assume without loss 
of generality that this is 1, 2, . . . , n. 

Let ao, ai, . . . , a„ be the labels of nodes at the beginning of the process. It 
is useful to first consider a deterministic version of the dynamics in question, 
specified as a game between two players 

— The first player is choosing one node to be scheduled. It is required that the 
sequence of nodes chosen by this player forms a periodic sequence tt. The 
goal of the first player is to prevent self-stabilization. 

— Given a node choice by the first player, the second player is responding with 
a choice of the second node to be scheduled. Unlike the first player, the 
sequence of nodes chosen by the second player can potentially vary between 
successive repetitions of the permutation tt. The goal of the second player is 
to make the system converge to state 0. 

The game above is an example of the scheduler-luck games from the self- 
stabilization literature [DIM95] . We will provide a strategy for the second player 
that (when applied) will turn any configuration into the "all zeros" configuration. 
But a winning strategy for the second player in the scheduler-luck game will 
be played with positive probability in any round of the scheduler. Thus with 
probability going to one (as the number of rounds goes to infinity) this strategy 
will be played at least in one round, making the system converge to state 0. 



The crux of the strategy is to carefuUy use the "partner node" of node 0, 
when this is scheduled, to create a segment of nodes 1, 2, ... i (with i nondecreas- 
ing, and eventuaUy reaching n) with labels zero at the beginning of a round of 
scheduling. 

This is simple to do at the very beginning: if node plays node 1 (when is 
scheduled), then the labels of the two node will be identical, thus when node 1 
is scheduled (and plays again node 0) the label of node 1 will be zero. 

If at the beginning of a round the label of node is 1 , we make it play (when 
scheduled at the beginning of a round) the node of smallest positive index (i + 1) 
still labelled 1. This will turn the labels of both nodes to 0. Further scheduling 
of nodes 1 to i + 1 will not change this, and at the end of the round, nodes 1 to 
i + 1 will still be labelled 0. 

If, on the other hand at the beginning of the round node is labelled 0, we 
make it keep this label (and, thus, not affect the zero labels of nodes 1 to 1) by 
making it play (when scheduled) against another node labelled (say node 1). 

To complete the argument it remains to show that for any configuration 
Xq, . . .Xn different from the "all zeros" configuration, in a finite number of rounds 
we will reach a configuration where the first case applies, and thus the length of 
the "all zero" initial segment increases. 

Indeed, assume that x(0) = and it stays that way throughout the process. 
Then, denoting by Yt = {x{l)t, . . . x{n)tY' the labels of the nodes 1 to n at the 
beginning of the i'th round, it is easy to see that the dynamics of the system is 
described by the recurrence 



Consider now B as a matrix over Z, rather than Z2. It is immediate to show 



The fc'th power over Z2 is obtained, of course, by reducing these values mod 
2. In particular define St to be the sum x{l)t + . . . x{n)t. It is easy to see that 
St = x{Q)t+i thus by our hypothesis St has to be zero. On the other hand a 
consequence of the previous result is that 



Yt+i = B-yt, 

with B — (bij) is a matrix of order n over Z2 specified by 




by induction that B'' 



{h\J) given by 





( mod 2). 



In particular 




By induction and algebraic manipulation we generalize this to higher order 
of iterated differences A''xt — A{A''~^xt) as: 



A'^xt 



i — k 



E{ n — i ^ t — 1 
t + k-i 



(mod 2). 



Let zq be the smallest index such that a;(io) = f. Then, by the previous 
relation 



A^-'Oxt 



El n — i + t ~ 1 



t + k-1 



n — ia + t — 1 
n — io + t — 1 



1 (mod 2). 



But this contradicts the fact that St = for every value of t and completes 
the proof. 



A 1-adaptive scheduler keeps repeating the nodes to choose according to a 
fixed permutation. Thus, for a fixed scheduler and interaction graph we can talk 
about the probability of stabilization in the limit. Also, for a given fixed sched- 
uler, the event that this limit is one is a deterministic statement. Consequently 
we can talk of the probability that this event happens when the interaction graph 
is sampled from a class of random graphs. As noted, for a random scheduler the 
condition that G has no isolated vertices is necessary and sufficient to guarantee 
self-stabilization with probability 1. This is also true for the adversarial model 
in the case of non-adaptive (f-fair) daemons. This is in case with the case of an 
edge daemon, when even non-adaptive daemons could preclude stabilization. 



Theorem 6 Let G be an instance of PDPD such that G has no isolated vertices. 
Then for any 1-fair node- scheduler and any initial configuration the system G 
reaches state with probability 1. 

The results of Theorem 5 should be contrasted with the corresponding result 
for edge schedulers, for which, as we showed, even non-adaptive daemons could 
preclude stabilization. 

The proof consists of the following three components: 

(i) our earlier result that guarantees a winning strategy for scheduler-luck game 
associated to the dynamics when the underlying graph is Sn , 

(ii) the partition of a spanning forest of G into node disjoint stars and 

(iii) the fact that the existence of such a winning strategy is a monotone graph 
property w.r.t to edge insertions. This is formally stated in the following 

Lemma 9. Suppose H is a graph such that a winning strategy W exists for the 
scheduler-luck game on a graph H . Let e ^ E{H), and let L = H U {e}. Then 
W is also a winning strategy for any graph L. 



Proof. Given any node ehoice by the first player, the second player can choose 
the corresponding node according to strategy W (thus never scheduling the 
additional edge e). The outcome of the game is, therefore, identical on H and L. 

Proof of Theorem 5. By Lemma 6 it is enough to show the existence of a 
winning strategy for the second player in the scheduler-luck game on a graph G, 
when G is a tree. We decompose tree G into a set {^i, . . . , Sp} of node-disjoint 
stars as follows. 

— Root G at an arbitrary node r. 

— Consider the star formed by the root and its children. Call it 5*1. 

— Remove the nodes in 5*1 and all edges with one end point incident on nodes 
in 5*1 

— Recursively apply the procedure on each forest created by the above opera- 
tion. 

Now consider a 1-fair schedule tt on graph G, corresponding to a strategy of 
the first player in the schedule- luck game on G. For every star Si, the projection 
TTi of the schedule on the nodes of Si (that amounts to only considering scheduled 
nodes that belong to Si) specifics a 1-fair schedule on Si. According to Theorem 
5, the second player has a winning strategy Wi for the scheduler- luck game on 
Si when the first player acts according to the schedule tt^. 

Next, we devise a strategy W for the scheduler-luck game on graph G, by 
by "composing" the winning strategies Wi. Specifically if the node chosen by 
the first player belongs to star S, strategy W will employ Wi to choose the 
corresponding second node. Since on each star Si the labels of the node will 
eventually be 0, IF is a winning strategy for the second player in the scheduler- 
luck game on G. □ 

6 Adaptive node schedulers 

Nonadaptive schedulers could not preclude self-stabilization. In contrast, as the 
following theorem shows, 3-fair nonadaptive node-schedulers are still powerful 
enough to preclude self-stabilization with complete certainty, and so are 2- fair 
adaptive^ schedulers. 

Theorem 7 The following are true: 

(i) Let the star graph Sn (Ki^n) be an instance of PDPD. Then there exists 
an initial configuration I and a 3-fair nonadaptive scheduler that precludes 
self- stabilization on Starn starting in I . 

(ii) Let the triangle be an instance of PDPD. Then there exists an initial con- 
figuration I and a 2-fair adaptive scheduler that precludes self- stabilization 
on K3 starting in L . 

® obviously, there are no 1-fair adaptive schedulers. 



Proof, (i) Consider the star graph Sn {Ki,n), with the center labeled and the 
rest of the nodes labeled 1,2, ... ,n. We have to provide an example of a 3-fair 
scheduler that precludes self-stabilization on some initial configuration. This 
initial configuration has two I's, at nodes f and 2. The scheduler repeats the 
schedule [0, 1, 1, 3, 4, ... n — 2, 2, 1, n — l,n ~ 1]. After the scheduling of 1 
1 the effect is that both nodes have label 0. Thus the scheduling of nodes 
3, 4, . . . , n — 2 does not change any label. With node 2 the label of node 
will change to 1, thus changing in the next step the label of node 1 back to 
1. Finally scheduling the node n — 1 twice turns back the label of node to 
0, thus yielding the initial configuration. It is easy to sec that the scheduler 
is 3-fair. 

(ii) Start with configuration / consisting of all ones. The scheduler will adap- 
tively schedule the nodes, in sequences of three, so that at the end of such 
a 3-block the system is guaranteed to be in configuration / again. Figure 1 
describes the strategy of the scheduler, assuming that node 1 is scheduled 
first. 
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Fig. 1. A round of the 2- fair adaptive scheduler 



Elements in the rectangle represent the state of the system, followed by the 
scheduled node (in square brackets), choices. Labels on the edges represent 
the possible probabilistic choices of the partner node, with multiple (inconse- 
quential) choices separated by a comma. Note that the scheduler has similar 
strategies if one of the nodes 2,3 is scheduled first. Also, note that a 3-block 
consists of either a permutation of a node, or two nodes, with the initial and 
the final node in the block being identical. The scheduler proceeds now to 
create an infinite schedule consisting of 3-blocks according to the following 
rule: 



(a) If a given block i? is a permutation then start the next block with the 
same starting clement as B. 

(b) Otherwise if the given block B is missing node z, start the next block 
by first scheduling z. 

It is easy to see that the scheduler we constructed is 2-adaptive and precludes 
self-stabilization. 

Although formal definition of the probability of self-stabilization is more 
complicated in this case, we can talk of the probability of self-stabilization for 
adaptive daemons as well. However, as we have seen, the result of Theorem 5 is 
no longer true: on stars, 1-fairness is stronger than 2-fair adaptive scheduling. 
It would seem that this result shows that nonadaptiveness is important for self- 
stabilization. However, we will see that the class of network topologies where 
this happens is reasonably limited. Indeed, we next study self-stabilization on 
Erdos-Renyi random graphs G{n,p). We will choose p in such a way that with 
high probability a random sample from G{n,p) has no isolated vertices^". In 
other words, we require that necessary condition on the topology of G holds 
with probability 1 — o(l). Call a graph G to be good if for any scheduler of 
bounded fairness and any starting configuration /, G starting at / converges to 
with probability 1 — o(l), as the number of steps goes to infinity. 

Theorem 8 Let p be s.t. np — logn oo. Then with probability 1 — o(l) a 
random graph G £ G{n,p) is good. 

Of course, a natural question is whether such a weakening of the original 
result, from any graph topology satisfying a given condition to a generic random 
graph satisfying the same condition, is reasonable. We are, however, not the 
first ones to propose such an approach. Indeed, except for a handful of cases the 
network that a given social dynamics takes place on is not known in its entirety. 
Instead, a lot of recent work (sec e.g. [PSV07,NBW06,Dur06] for presentations) 
has resorted to the study of generic properties of random network models that 
share some of the observable properties of a fixed network (such as the Internet 
or the World Wide Web. 

Proof. The plan of the proof is similar to that for 1-fair schedulers. Define a 
round of a 5- fair scheduler to consist of a consecutive sequence of b{n — 1) + 1 
steps. 

(i) We prove that for graphs from a class B of "base case" graphs (Lemma 7 
below) the second player has a winning strategy in the scheduler-luck game 
associated to any scheduler of bounded fairness, where the games corresponds 
to a finite number of rounds of the scheduler. 

(ii) We use the monotonicity of the existence of a strategy. 

^° A random sample from G{n,p) has no isolated vertices with probability 1 — o(l) 
when [JLROO] np — logn — > oo. 



(iii) We show that with probabihty 1 — o(l) the vertices of a random sample 
graph G from the graph process can be partitioned such that aU the induced 
subgraphs are isomorphic with one graph in G. 

Lemma 10. The following are true: 

(i) Let G be a graph with a perfect matching. Then for any scheduler S of 
hounded fairness, and any initial configuration on G, the second player has 
a winning strategy for the scheduler-luck game corresponding to one round 
of the scheduler. 

(ii) Let the line L„, n > 6 be an instance of PDPD. Then for any node- scheduler 
S of hounded fairness and every initial configuration I , the second player has 
a winning strategy in the scheduler-luck game associated with two consecutive 
rounds of S. 

Proof, (i) A perfect matching M of G specifies a winning strategy in a scheduler- 
luck game: each node plays (when scheduled) against its partner in Af . Since 
every node is scheduled at least once in a round of scheduling, every edge of 
M is played at least twice. Therefore, irrespective of the initial configuration, 
the final configuration is 0. 

(ii) The lemma only needs to be proved in the case when n is odd (in the 
other case the strategy based on perfect matchings applies). In this case 
the winning strategy is specified as follows: 

(a) In the first round turn the leftmost ^4 portion of i„ into the all zero 
state by playing the matching based winning strategy. 

(b) In the second round nodes in the leftmost L3 will only choose to play 
against each other when scheduled, thus remaining at 0. The remaining 
nodes form a graph isomorphic to Ln-3, and in this round we use the 
perfect matching based strategy for this graph. 

We now note now that the statement of Lemma 6 extends to scheduler-luck 
games associated to node daemons of bounded fairness. The proof is similar (a 
strategy for the game on G is also a strategy for the game on a graph with a 
larger set of edges) . Theorem 7 immediately follows if n is even: a classical result 
in random graph theory (see e.g. [JLROO] pp. 82-85) asserts that with probability 
1 — 0(1) G will have a perfect matching. 

To complete the proof of Theorem 7 we only need to deal with the case when 
n is odd. For a graph G and a set of vertices V denote by G]v the subgraph 
induced by vertex set V. 

Lemma 11. With probability 1 — o(l) G can be partitioned into V = Vi U V2 
such that: 

(i) G\vi contains L7 as an edge-induced subgraph. 

(ii) G\v2 is a graph with a perfect matching. 



Theorem 7 follows from Lemma 8, since for both G|vi and G\v2 the second 
player has a winning strategy in the schcdulcr-luck game. A winning strategy for 
the corresponding game on G proceeds by using the winning strategy for Gjyj, 
when the scheduled node is in Vi and the winning strategy for Gji/^ when the 
scheduled node is in V2. 

The proof of Lemma 8 goes along lines similar to that of the proof of the 
existence of a perfect matching in a random grapg(see [JLROO] pp. 82-85). The 
first step is to show that w.h.p. G does not contain a set of distinct vertices 
xq, X2, ^3, X4 such that: 

- degcixo) = degcixi) = 1. 

— For every i = 0, 3, Xi and Xi+i are adjacent. 

This is easy to see, since the expected number of such structures is 0{n^p'^{l — 

Consider now a random graph G, conditioned on not containing such a struc- 
ture, and a vertex xq in G of degree one. Since this information only exposes 
information on the edges with one endpoint at xq, with probability 1 — o(l) 
graph H = G \ {xq} has a perfect matching. Let Xi be the node in G \ {a;o} 
adjacent to xq, and let X2 be the node matched to Xi in H. With probability 
1 — 0(1) X2 has another neighbor X3 in H, (otherwise G would contain a cherry, 
i.e. two vertices of degree one at distance exactly 2 in G (see Figure 2). But (see 
[JLROO] pp. 86) a random sample from G{n,p) only contains a cherry with prob- 
ability 0(1). Let Xi be the node X3 is matched to in H. Again, with probability 
1 — 0(1) X4 has a neighbor X5 in H different from X3 (otherwise the five vertices 
Xq to Xi would form a structure we have conditioned on not occurring in G). 
Finally let xq be the node matched to X5 in H. Then the restriction of G to the 
set Vi = {xq, . . . ,xe} contains a copy of L7, and G restricted to V2 ~ V \ Vi 
contains a perfect matching (induced by the perfect matching on H). 




Fig. 2. A cherry in a graph (with bold lines) 



7 Speed of convergence 



The previous theorems have shown that results concerning convergence to a fixed 
point can be studied in (and extend to) an adversarial framework. Perhaps what 
is not preserved as well in the adversarial framework is results on the computa- 
tional efficiency of convergence to equilibrium. Such results include, for instance, 
the above mentioned O(nlogn) bound of [DGG+02]. The proof of this theorem 
displays an interesting variation on the idea of a potential function. It uses such 
a function, but in this case the value of the function only diminishes "on the av- 
erage" , rather than for every possible move. Therefore bounding the convergence 
time seems to critically use the "global" randomness introduced in the dynamics 
by random matching, and does not trivially extend to adversarial versions. On 
the other hand, the proof of Theorem 4 only guarantees an exponential upper 
bound on expected convergence time. 

We have investigated experimentally the convergence time on C„ for some 
classes of 1-fair schedulers (permutations). Some of our results are presented 
in Figure 3, where we present the average number of rounds, rather than steps, 
over 1000 samples at each point. The symbol id denotes the identity permutation 
(12 . . . n), p3 is the permutation a[i] = 3z(mod n), (13) refers to permutations 
with pattern (13245768 . . .), and rd refers to the maximum average number of 
rounds, taken over 10 random permutations. In all cases the convergence time is 
consistent with the above-mentioned O(nlogn) result. 
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Fig. 3. Rounds on C„ under 1-fair scheduling. 



We arc unable to obtain such a result (and leave it as an interesting open 
problem) It is even more interesting to study the dependency of the mixing 
time of the dynamics [DGG+02] on the underlying network topology. While 
there are superficial reasons for optimism (for some models in evolutionary game 
theory, e.g. [MorOO], the impact of network topology on the convergence speed 

A promising approach is outlined in [FM05] 



of a given dynamics is reasonably well understood), the reader is directed to 
[MR06] (especially the concluding remarks) for a discussion on the difficulties 
of connecting network topology and convergence speed for the specific dynamics 
we study. 

8 Discussion of Results and Conclusions 

We have advocated the study of evolutionary game-theoretic models under ad- 
versarial scheduling, similar to the ones in the theory of self-stabilization. As an 
illustration we studied the Iterated Prisoners'Dilemma with the win-stay lose- 
shift strategy. 

Our results are an illustration of the adversarial approach as follows: 

(i) Start with some result P, valid under random scheduling. The orig- 
inal statement is presented in Subsection 2.2. 

(ii) Identify several structural properties of a random scheduler that 
impact the validity of P. The random scheduler is 

— /air more precisely 0{n logri) fair w.h.p. by the Coupon Collector Lemma. 

— endogeneous, since the next edge to be scheduled is not fixed in advance. 

— nonadaptive, since the next edge to be scheduled does not depend on the 
configuration of the system. 

(iii) Identify those properties (or combinations of properties) that are 
necessary /sufficient for the validity of P. 

Theorem 1 shows that fairness is a necessary condition for the extension of 
the original result to adversarial settings. Next, the definition of node and 
edge schedulers illustrates another important property of random schedulers: 
endogeneity of agent interactions: an edge scheduler completely specifics the 
dynamics of interaction. In contrast, node schedulers provide perhaps the 
weakest possible form of endogeneity: the underlying social network is still 
fixed, but the agents can choose a neigbor among his neighbors to interact 
with (or simply play a random one). 

Theorems 4 and 5 show that, in contrast with the case of edge schedulers, 
even this limited amount of endogeneity is sufficient to recover the original 
result for random scheduler. Moreover, the proofs illuminate the role of en- 
dogeneity, that was somehat obscured in the (trivial) original proof that the 
Pavlov dynamics (under random matching) converges with high probability 
to the "all zeros" fixed-point. This proof implicitly relies on the fact that 
from every state there exists a sequence of "right" moves, that "funnels" the 
system towards the fixed point. For node schedulers the existence of a such 
a set of right moves is proved by explicit construction and is more difficult 
in the adversarial setting. The existence of such a set of moves is precisely 
what exogeneous choice of agents is able to preclude. 



(iv) Correspondingly, identify those properties that are inessential to 
the validity of P. In the process one can reformulate (if needed) 
the original statement in a way that makes it more robust. 

Theorem 6 shows that, if we allow schedulers to be adaptive, then net- 
work topology becomes important, and can invalidate the original result in 
an adversarial setting. However adaptiveness (or, equivalently, the amount of 
fairness) is inesentialifwe require the convergence result to only hold generi- 
cally with respect to the class of network topologies described by Erdos-Renyi 
random graphs. 

The results we proved also highlight a number of techniques from the theory 
of self-stabilization that might be useful in developing a general theory: 

— the concept of schcdulcr-luck game. 

— composition of strategics by partitioning the interaction topology. 

— monotonicity and "generic preservation" via threshold properties. 

Obviously, a reconsideration of more central game-theoretic models under 
adversarial scheduling is required (and would be quite interesting) . 
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